Addressing Hospital Staffing Shortages

Several hospitals face nurse staffi ng shortages for surgeries. This research focuses on building a system with a Baxter robot capable of identifying surgical tools using computer vision and delivering them to the surgeon on demand. This would deal with the issue of nurse unavailability during simple surgical procedures. The key aspects of the project were testing the accuracies of various artifi cial neural networks (ANNs) in classifying surgical instruments and programming Baxter to implement a surgical tool delivery system using magnets at the tip of its 7 degrees of freedom (DOF) robotic arms. The methodology consisted of fi rst implementing algorithms to enable Baxter to do pick and deliver tasks for surgical tools, and second, gathering HuMoments of various tools using the cameras on Baxter's arm, which were then used to train the ANNs. Tool detection accuracies of ANNs with hidden layer neuron number varying from 5–50 and learning rates varying from 0.005‒0.1 were collected. Then, the tool identifi cation and tool delivery system were merged together to create a turn-byturn dynamic tool tracking and delivery system, which retrieved tools, based on the surgeons input, through a Leap Motion Controller. In addition to delivery, the system was modifi ed to retrieve used tools from the surgeon, using a computer visionbased approach. The optimal ANN confi guration consisted of an ensemble of various ANNs working together and achieved a detection accuracy of 93%. The average time taken for a mock abdominal incision surgery with the system is expected to be around 10 minutes and 30 seconds.